Regularization is a technique to solve the problem of overfitting in a machine learning algorithm by penalizing the cost function. It does so by using an additional penalty term in the cost function. So, how can L2 Regularization help to prevent overfitting? Let's first look at our new cost function: It controls the trade-off between two goals: fitting the training data well vs keeping the params small to avoid overfitting. The regularization term will heavily penalize large wᵢ.

In this post we will talk about applying gradient descent on \(m\) training examples. Now the question is how we can define what gradient descent is? A gradient descent is an efficient optimization algorithm that attempts to find a global minimum of a function. It also enables a model to calculate the gradient or direction that the model should take to reduce errors (differences between actual \(y\) and predicted \(\hat{y}\)). Now let's remind ourselves what the cost function is?